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DETAILED ACTION 



Claim Rejections - 35 USC § 102 



1 . The following is a quotation of the appropriate paragraphs of 3 5 U. S. C. 1 02 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in a patent granted on an application for patent by another filed in the United 
States before the invention thereof by the applicant for patent, or on an international application by another who 
has fulfilled the requirements of paragraphs (1). (2). and (4) of section 371(c) of this title before the invention 
thereof by the applicant for patent. 

The changes made to 35 U.S.C. 102(e) by the American Inventors Protection Act of 1999 
(AIPA) do not apply to the examination of this application as the application being examined 
was not (1) filed on or after November 29, 2000, or (2) voluntarily published under 35 U.S.C. 
122(b). Therefore, this application is examined under 35 U.S.C. 102(e) prior to the amendment 
by the AIPA (pre- AIPA 35 U.S.C. 102(e)). 

2. Claims 1-25 are rejected under 35 U.S.C. 102(e) as being anticipated by Xiong et al. U.S. 
Patent No. 6,434,265. 

3. Claim 1 recites an image synthesis method. The Xiong reference discloses a method for 
constructing a panorama from rectilinear images in 3D through projective registration and 
caUbration including: (1) the projective registrations of overlapping images, (2) calibration and 
global optimization of these images, a self calibration in which 2D image planes are positioned 
as 3D planes in space, and (3) the composing or blending in which images are ready to be re- 
projected to a 3D environment map with pixels in overlap regions being composed from multiple 
images (column 4, lines 5-40). The Xiong reference further teaches that overlapping photographs 
are analyzed to determine what orientation the photographs were taken in order to establish a 
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common ground for subsequent operations and the panorama is constructed on a particular 
geometry that will best facilitate the rendering of the projection of the panorama onto a chosen 
viewing plane for viewing. The Xiong reference implicitly discloses some typical geometry on 
which panoramas are formed (column 8, lines 18-58). 

Claim 2 recites all the limitations of claim 1 and adds the Umitation of "a focal length 
obtaining step." The Xiong reference teaches finding projective parameters such as 3D rotation 
parameters (pan, tilt roll), center of projection of images, ratio of focal lengths, and the like 
(column 10, lines 9-28). The Xiong reference implicitly teaches finding the camera internal 
parameters vector of which the focal length is a component (column 1 1, lines 15-42). 

Claim 3 recites all the limitations of claim 1 and adds the limitation of "a changing step 
of changing the mapping mode." The Xiong reference teaches that panorama is constructed on a 
particular geometry that will best facilitate the rendering of the projection onto a chosen viewing 
plane such as cubic, polyhedral, cylindrical and spherical geometries (column 8, Unes 18-58). 

Claim 4 recites all the limitations of claim 1 and adds the Umitation of "issuing a 
warning." The Xiong reference teaches in figures 2 and 3 a user interface and a global 
optimization that provides feedback to the computer system such as issuing warning messages on 
the computer monitor 218 when the pair-wise objective function is not desirable for a poor 
selection of the projection viewing plane (and the resulting panoramas will have imperfectly 
aligned images that give shadow or ghosting effects) due to a poor selection of the projection 
viewing plane. The resulting panoramas will have imperfectly aligned images that give shadow 
or ghosting effects. Nevertheless, the Xiong reference points to a seamless multi-resolution 
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average blending method that would result in an absent of shadow effects (column 14, lines 1- 
45). 

Claim 5 recites all the limitations of claim 1 and adds the limitation of "a displaying step 
of displaying a cuttable rectangular region." The Xiong reference teaches how to align images 
more precisely by changing the coordinates for positioning an image. The Xiong reference 
further teaches placing the images 1210 at selected tangents on the viewing sphere 1220 (figure 
12, and column 17, lines 12-65). 

4. Claim 6 recites "an image synthesis apparatus. The Xiong reference teaches in figure 3 an 
apparatus for image synthesis. 

Claim 7 recites all the limitations of claim 6 and adds the limitation of "a focal length 
obtaining step." The Xiong reference teaches fmding projective parameters such as 3D rotation 
parameters (pan, tih roll), center of projection of images, ratio of focal lengths, and the Uke 
(column 10, Hnes 9-28). The Xiong reference implicitly teaches finding the camera internal 
parameters vector in his image synthesis apparatus of figure 3 because the focal length is a 
component of that vector (column 1 1, lines 15-42). 

Claim 8 recites all the limitations of claim 6 and adds the limitation of "a changing step 
of changing the mapping mode." The Xiong reference teaches that panorama is constructed on a 
particular geometry that will best facilitate the rendering of the projection of the panorama onto a 
chosen viewing plane such as cubic, polyhedral, cylindrical and spherical geometries (column 8, 
lines 18-58). 

Claim 9 recites all the limitations of claim 8 and adds the limitation of "issuing a 
warning." The Xiong reference teaches in figures 2 and 3 a user interface and a global 
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optimization that provides feedback to the computer system such as issuing warning messages on" 
the computer monitor 218, e.g., when the pair- wise objective function is not desirable (and the 
resulting panoramas will have imperfectly ahgned images that give shadow or ghosting effects) ' 
due to a poor selection of the projection viewing plane. The resuhing panoramas will have 
imperfectly aligned images that give shadow or ghosting effects. Nevertheless, the Xiong 
reference points to a seamless multi-resolution average blending that is absent of shadow effects 
(column 14, lines 1-45). 

Claim 10 recites all the limitations of claim 8 and adds the limitation of "a displaying step 
of displaying a cuttable rectangular region." The Xiong reference teaches how to align images 
more precisely by changing the coordinates for positioning an image. The Xiong reference 
further teaches placing the images 1210 at selected tangents on the viewing sphere 1220 (figure 
12, and column 17, lines 12-65). 

5. Claim 1 1 recites a computer-readable storage medium having a program for 
implementing image synthesis method. The Xiong reference teaches a program residing in 
system memory 220 which stores output data and other data (column 3, lines 54-67). 

Claim 12 recites all the Umitations of claim 1 1 and adds "a focal length obtaining step." 
The Xiong reference teaches finding projective parameters such as 3D rotation parameters (pan, 
tih roll), center of projection of images, ratio of focal lengths, and the like (column 10, lines 9- 
28). The Xiong reference implicitly teaches finding the camera internal parameters vector of 
which the focal length is a component (column 1 1, lines 15-42). 

Claim 13 recites all the limitations of claim 1 1 and adds the limitation of "a changing step 
of changing the mapping mode." The Xiong reference teaches that panorama is constructed on a 



Application/Control Number: 09/624,385 Page g 

Art Unit: 2672 

particular geometry that will best facilitate the rendering of the projection of the panorama onto a 
chosen viewing plane such as one of the cubic, polyhedral, cylindrical and spherical geometries 
(column 8, lines 18-58). 

6. Claim 14 recites an image synthesis method comprising a generating step of generating 
coordinate-space transformation parameters. The Xiong reference teaches for example in a local 
pair-wise registration the estimation of parameters in a projective matrix which represents a 
coordinate transformation matrix (column 11, lines 1-46). 

Claim 15 recites all the limitations of claim 14 and adds the limitation of "a changing 
instruction step of issuing a mapping mode changing instructions." The Xiong reference teaches 
that panorama is constructed on a particular geometry that will best facilitate the rendering of the 
projection of the panorama onto a chosen viewing plane such as cubic, polyhedral, cylindrical 
and spherical geometries (column 8, Unes 18-58). The Xiong reference further teaches 
calibration as the second major step in authoring panoramas to extract camera internal and 
external parameters from those projective matrices (column 12, lines 33-44). The Xiong 
reference discloses that the improved projective matrix parameters will in turn be used to 
generate improved estimation of camera parameters and blending may be further iterated after an 
initial blending (column 13, lines 53-67). 

7. Claim 16 recites an image synthesis method comprising a reference position-setting step. 
The Xiong reference teaches a multi-resolution weighted average blending for perspective 
alignment (column 14, lines 2-45). The Xiong reference also teaches for example in column 1 1 
setting or estimating camera orientation and the rotation matrix that is based on camera 
orientation. 
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8. Claim 17 recites all the limitations of claim 14 and adds the limitation of "a storage step 
of storing a generated panoramic synthesized image." The Xiong reference teaches panoramic 
canvas which can be simply be a buffer or memory in a computer system such as memory 220 of 
figure 2 and onto which the blended images is copied (column 15, lines 15-28). 

9. Claim 1 8 recites an image synthesis apparatus comprising a generating means for 
generating coordinate-space transformation parameters and image synthesis means for 
combining the images. The Xiong reference teaches for example a pair-wise registration module 
222 and the projection function module 228. The parameters in a projective matrix are estimated 
which represents an image coordinate transformation matrix. The projection function module 
228 constructs a panoramic scene by projecting the blended image onto any designated geometry 
view surface (column 4, hnes 21-52, and column 1 1, lines 1-46). 

Claim 19 recites all the limitations of claim 18 and adds the limitation of "a changing 
means for changing the mapping mode." The Xiong reference teaches a projection module 228 
residing in memory 220 and operating the processor 212 may be controlled through the user 
interface 230 to allow a user to select what geometry will be projected onto (i.e., changing the 
mapping mode) and to control and modify other factors, including the use of photo re-touching 
software such as PhotoShop for modifying the final panorama (column 4, lines 41-49). 

Claim 20 recites all the limitations of claim 18 and adds the limitation of "a reference 
position setting means." The Xiong reference teaches for example in column 1 1 setting or 
estimating camera orientation on which the rotation matrix is based. 

Claim 21 recites all the limitations of claim 18 and adds the limitation of "storage means 
for storing a generated panoramic synthesized image, coordinate transformation parameters and 
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coordinate-space transformation parameters " The Xiong reference teaches storage means such 
as panoramic canvas which can be simply a buffer or memory in a computer system such as 
memory 220 in figure 2 and onto which the blended images is copied (column 15, lines 15-28). 
The Xiong reference teaches that the improved projective matrix parameters (i.e., the coordinate 
and coordinate-space transformation parameters) can be used to generate improved estimation of 
camera parameters and blending may be further iterated after an initial blending (column 13, 
lines 53-67). Finally, the Xiong reference teaches projective parameters, e.g., 3D rotation 
parameters, center of projection of images, ratio of focal lengths, and an affine transformation in 
constructing a two-dimensional view of the environment (column 10, lines 16-28), 

10. Claim 22 recites a computer-readable storage medium having a program for 
implementing image synthesis method and a generating step of generating coordinate-space 
transformation parameters. The Xiong reference teaches a program residing in system memory 
220 which stores output data and other data (column 3, lines 54-67). The Xiong reference also 
teaches for example in a local pair- wise registration the estimation of parameters in a projective 
matrix which represents a coordinate transformation matrix (column 1 1, lines 1-46). Finally, the 
Xiong reference teaches that the improved projective matrix parameters will in turn be used to 
generate improved estimation of camera parameters and blending may be further iterated after an 
initial blending (column 13, Unes 53-67), 

1 1 . Claim 23 recites an image synthesis method comprising an editing step and a storage 
step. The Xiong reference teaches a user interface that has fields for any and all internal and 
external parameters of the projection matrix of the images including aspect ratio, number of rows 
of images, the tilt between rows, the angle between photos within a row, the roll of each image 
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taken, image center position, focal length, camera orientation, the brightness and contrast of 
image, etc. The Xiong reference further teaches that a user may have the ability to adjust 
parameters for images captured with a particular methodology (column 9, lines 1-21). The Xiong 
reference also teaches panoramic canvas which can be simply a buffer or memory in a computer 
system such as memory 220 shown in figure 2 and onto which the blended images is copied 
(column 15, lines 15-28). 

Claim 24 recites all the limitations of claim 23 and adds the limitation of "a mapping 
mode is changed." The Xiong reference teaches that panorama is constructed on a particular 
geometry that will best facilitate the rendering of the projection of the panorama onto a chosen 
viewing plane such as cubic, polyhedral, cylindrical and spherical geometries (column 8, lines 
18-58), 

Claim 25 recites all the limitations of claim 23 and adds the limitation of "an adjoining 
part of the image is corrected in editing step." The Xiong reference teaches determining the 
boundary of overlap regions in multi-resolution blending (column 14, lines 46-62). The Xiong 
reference also teaches a flowchart for the blending procedure resulting in a desired smooth effect 
and a grassfire transform on the panoramic canvas for the blending of more than two images 
(column 16, hnes 40-67). 

Conclusion 

12. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

a. SzeUski et al. U.S. Patent No. 6,097,854 discloses an image mosaic construction 
system and apparatus with patch-based alignment. 
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b. Kumar et al. U.S. Patent No. 5,963,664 discloses a system for generating three- 
dimensional mosaics from a plurality of input images. 

13. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Jin-Cheng Wang whose telephone number is (703) 605-1213. 
The examiner can normally be reached on 8:00 AM - 4:30 PM. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mike Razavi can be reached on (703) 305-4713. The fax phone numbers for the 
organization where this application or proceeding is assigned are (703) 308-6606 for regular 
communications and (703) 308-6606 for After Final communications. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 395-3900. 

jew 

September 25, 2002 




SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 



